An Empirical Study of Smoothing Techniques for LanguageModelingStanley

We present an extensive empirical comparison of several smoothing techniques in the domain of language modeling, including those described by Jelinek and Mer-cer (1980), Katz (1987), and Church and Gale (1991). We investigate for the rst time how factors such as training data size, corpus (e.g., Brown versus Wall Street Journal), and n-gram order (bigram versus trigram) aaect the relative performance of these methods, which we measure through the cross-entropy of test data. In addition, we introduce two novel smoothing techniques , one a variation of Jelinek-Mercer smoothing and one a very simple linear interpolation technique, both of which out-perform existing methods.